Modulation spectral features for speech emotion recognition using deep neural networks

This work explores the use of constant-Q transform based modulation spectral features (CQT-MSF) for speech emotion recognition (SER). The human perception and analysis sound comprise two important cognitive parts: early auditory cortex-based processing. considers spectrogram-based representation whereas includes extraction temporal modulations from spectrogram. spectrogram is called feature (MSF). As (CQT) provides higher resolution at salient low-frequency regions speech, we find that CQT-based spectrogram, together with its modulations, a enriched emotion-specific information. We argue CQT-MSF when used 2-dimensional convolutional network can provide time-shift invariant deformation insensitive SER. Our results show outperforms standard mel-scale on popular SER databases, Berlin EmoDB RAVDESS. also our proposed shift scattering coefficients, hence, showing importance joint hand-crafted self-learned instead reliance complete features. Finally, perform Grad-CAM to visually inspect contribution over